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Abstract 

Principal preparation programs are designed to ensure that candidates who successfully complete the 
programs are qualified and knowledgeable, and have had leadership experiences that prepare them to 
compete for leadership positions in today’s schools (Hale & Moorma, 2003). Providing meaningful leader¬ 
ship experiences in non-internship programs can be a challenge. The purpose of this study was to evaluate 
the impact of candidates’ field experiences in leadership based on a locally developed pre and post self 
assessment. The instrument consisted of 54 items developed around the California Professional Standards 
for Educational Leaders (CPSEL). Results indicated significant differences in candidates’ perceptions of 
their pre and post levels of experience as evidenced by mean score and matched case comparisons. The 
impact of entry level years of experience was examined. Implications and future program enhancements 
based on the results are included. 
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Sumario en espanol 

Principales programas de la preparation son disenados para asegurar que candidates que completen exitosa- 
mente los programas son calificados e informado, y han tenido experiencias de liderazgo que preparan ellos 
competir para posiciones de liderazgo en las escuelas de hoy (Arrastre y Moorma, 2003). Proporcionar expe¬ 
riencias significativas de liderazgo en programas de no-puestos de interno pueden ser un desafio. El proposito 
de este estudio fue de evaluar el impacto de experiencias del campo de candidatos en el liderazgo basado 
en un localmente desarrollado pre y anuncia auto evaluation pre y anuncia auto evaluation. El instrumento 
consistio en 54 articulos desarrollados alrededor de la California Estandares Profesionales para Lideres Ed- 
ucativos (CPSEL). Los resultados indicaron diferencias significativas en las percepciones de candidatos de su 
pre y niveles de poste de experiencia como demostrado por la c.uenta mala y comparaciones emparejadas de 
caso. El impacto de la entrada anos que pianos de experiencia fueron revisados. Las implicaciones y futures 
aumentos del programa basados en los resultados son incluidos. 

note: Esta es una traduction por computadora de la pagina web original. Se suministra como 
information general y no debe considerarse completa ni exacta. 


1 Introduction 

“I can’t get a job without experience; 

I can’t gain experience without a job.” 

The proverbial Catch 22. 

Knowledge versus experience is often a consideration in employment decisions regardless of the field. When it 
comes to educational leadership, finding qualified and knowledgeable candidates with credentials is much eas¬ 
ier than finding those with experience in leadership activities necessary in modern schools (Hale & Moorma, 
2003). The best predictor of what a person will do in the future is what they have done in the past. Practice 
may not make perfect, but it improves the odds for eventual success. But how, in a non-internship program, 
does one practice being a school leader when one’s range of experience is most often limited to classroom 
teaching? 

The opportunity and design of meaningful experiences can vary; whether through internships or fieldwork, 
situated learning in authentic workplaces has been the hallmark of exemplary school leadership preparation 
programs (Browne-Ferrigno, 2003; Jackson & Kelly, 2002; Perez, Uline, Johnson, James-Ward, & Basom, 
2011). In a year-long study of 18 students participating in a principal licensure program, Browne-Ferrigno 
(2003) found that informal and informational experiences helped participants frame their concept of the 
principalship; “...the purpose of field-based learning guided by leadership practitioners is to begin initial 
socialization into a new community of practice (p. 495).” 

Many notable principal preparation programs assign concurrent course-specific, field experiences within 
classes across the duration of programs so that students acquire practical experience (Dishman & Redish, 
2011; Jackson & Kelly, 2002; Kirkpatrick, 2000; Orr, 2006; Perez, et al., 2011). Researchers have found that 
it is through immersion in real school problems that preservice principals gain meaningful problem solving 
skills (Browne-Ferrigno & Muth, 2004; Browne-Ferrigno, 2007; Perez, et al., 2011). In their study to analyze 
changes to a principal preparation internship program, Risen and Tripses (2008) found that expert problem 
solving was central to strengthening future school leaders. 

1 http://www.ncpeapublications.org 
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The purpose of principal preparation programs is to develop leadership skills and capacities within pre¬ 
service principals (Dishman & Redish, 2011; LaPointe & Davis, 2006; Orr, 2010). A key component of this 
preparation is administrative experience that reflects what actually occurs in today’s schools. In California, 
Preliminary Administrative Services credential programs must meet 15 Standards for accreditation. Stan¬ 
dard 7 - Nature of Field Experiences, and Standard 8 -Guidance, Assistance and Feedback, provide criteria 
for planning and evaluating required program elements. Standards 10-15, closely aligned with the California 
Professional Educational Leadership Standards (CPSELs), describe the skills, knowledge, and dispositions 
expected of entry-level administrators. 

The fieldwork experiences component designed and facilitated by the Department of Educational Lead¬ 
ership at California State University, Fullerton (CSUF) consists of four semester units spread throughout 
the five-term program. Instructors guide students in the development of fieldwork plans, which identify 
specific leadership tasks and responsibilities to be completed under the direction of their mentors, who are 
experienced administrators, typically principals or assistant principals at their sites. Activities must align 
with the CPSELs, and during their final semester, students submit a portfolio of artifacts to support their 
written analysis of their experience and progress toward competency in each of the standards. 

In 2008, CSUF Educational Leadership faculty convened a task force to review the department’s fieldwork 
standards and procedures. Faculty responsible for evaluating administrative competency had expressed 
concern that the leadership experiences that students plan and complete are often the result of convenience 
and opportunity, rather than need. Although students collect substantial evidence regarding their fieldwork 
accomplishments and are able to articulate the alignment with the leadership standards, faculty questioned 
whether the activities addressed the wide range of skills and expertise demanded of today’s educational 
leaders. 

Instructors concluded that fieldwork may need to be more prescribed and individualized, congruent with 
the California Professional Standards for Educational Leadership (CPSELs) but also designed to expand the 
experiential base of each student. Students enter the administrative credential program with varied educa¬ 
tional and professional backgrounds. Some are veteran educators, with 20 or more years in the profession 
while others are neophytes, with as few as three years of experience. Some are department chairs, team 
leaders, or teachers on special assignment (TOSAs) while others have developed expertise in a specific area 
of instruction, such as special education. Fieldwork, we concluded, should force students to move out of their 
comfort zones and into new arenas of experience. Toward that goal, we developed a tool. The Candidate 
Inventory of Personal Leadership Competence was designed to help students identify their areas of relative 
strengths and weaknesses, an assessment that could be used to drive their fieldwork plans and to measure 
growth. The purpose of this study was to evaluate the impact of candidates’ field experience in leadership 
based on the pre and post implementation of that tool. 

2 Research Questions 

1. What degree of change occurred in students’ pre and post self-assessment of their level of experience 
on each of the CPSELs, and how similar or different were the changes from pre to post self assessment 
among the six standards? 

2. Did the degree of reported gain/loss in leadership experience vary among individual students? 

3. How similar or different were the changes from pre to post self assessment according to the total years 
of professional work experience (TYE) of the candidates? 


3 Methodology 

This study is based on an analysis of candidates’ pre and post self assessment of their own level of experience 
in activities aligned with the six CPSELS. The instrument, The Candidate Inventory of Personal Leadership 
Competence, was designed around the six CPSELs which state that an instructional leader promotes the 
success of every student by... 
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1. Facilitating the development, articulation, implementation, and stewardship of a vision of learning that 
is shared and supported by all stakeholders. 

2. Advocating, nurturing, and sustaining a school culture and instructional program conducive to student 
learning and staff professional growth. 

3. Ensuring management of the organization, operation, and resources for a safe, efficient, and effective 
learning environment. 

4. Collaborating with faculty and community members, responding to diverse community interests and 
needs, and mobilizing community resources. 

5. Acting with integrity, fairness and in an ethical manner. 

6. Understanding, responding to, and influencing the political, social, economic, legal, and cultural con¬ 
text. 

Collaboratively, we “unwrapped” the standards and operationalized each one in terms of specific leadership 
behaviors and experiences. We asked ourselves, “What, specifically, do leaders need to be able to do?” 
Second, “What experiences and knowledge do our students already have and which do they lack?” 

At its inception, we believed that this forced-choice instrument would accomplish three things: (a) 
familiarize students with the standards and program expectations, (b) identify a starting place for developing 
meaningful fieldwork targeted to the areas of greatest need, and (c) provide baseline data to measure perceived 
growth and program effectiveness. 

The pre assessment was implemented in Fall 2008, and the first post-assessment took place in Spring 
2010 when those initial cohorts of participating students completed the preparation program. Each pencil 
and paper assessment was administered within the confines of a classroom, directed by the fieldwork course 
instructor. 

The Candidate Inventory of Personal Leadership Competence consists of six pages, one page devoted to 
each CPSEL with a total of 58 items. For each of the professional educational standards, there are 9-11 
specific leadership activities or roles for which students are to indicate their current level of preparation based 
on two dimensions: experience and knowledge. Responses were on a four-point scale, with 4 representing 
“high” and 1 representing “low.” The activities listed under each standard were collected from various print 
and online sources. Examples of actual specific leadership activities by CPSEL Standard contained in the 
Inventory include the following: 

1. Vision of learning: 

• Convene a group of teachers to evaluate progress toward the school mission. 

• Evaluate the effectiveness of the reading (or math) intervention program. 

2 . Culture, instructional program: 

• Assess the rigor or coherence of the curricular program at my school 

• Develop an induction plan for new teachers 

3. Organizational management. . . effective learning environment: 

• Advocate for increased resources 

• Facilitate the school safety task force 

4. Collaboration.. . diverse community needs: 

• Develop and disseminate a community newsletter to diverse parent and community groups, agencies, 
and businesses 

5. Fairness, integrity... ethics: 

• Analyze an employee grievance relative to an instructional issue 
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• Research and present the contents of the Family Educational Rights and Privacy Act (FERPA)to 
faculty, staff, and parents 

6. Influencing political, social, economic, legal, and cultural context: 

• Critique an educational policy 

• Interview local and state politicians to understand their positions and seek support for the improvement 
of teaching and learning in your school/district 

These examples represent only 11 of the 58 items contained in The Candidate Inventory of Personal Lead¬ 
ership Competence. 

3.1 Population 

The population consisted of 54 female and 28 male educators.Table 1 describes the 82 administrative candi¬ 
dates according to years of experience at the point they began their administrative credential preparation. 
Table 1 

Numbers of Participants by Teaching Assignment and Years of Experience Upon Enrollment 




Total Years Experience (TYE) 


Assignment 

< 5 Years 

5-9 Years 

10 or more Y rs 

Total 


n 

n 

n 

n 

Elementary' School 

15 

7 

s 

30 

Mddle School 

Z 

9 

1 

12 

High School 

15 

11 

9 

35 

District 


1 

Z 

3 

Other 

1 


1 

Z 

Total 

33 

28 

21 

82 


As shown in Table 1, the distribution of students was scattered, with the greatest number (33) reporting 
fewer than five TYE. The majority of students had five years or more TYE, with 21 reporting 10 or more 
years. Elementary and high school teachers were similar in representation (30 and 35, respectively); three 
students reported assignments at the district level. 

3.2 Data Analysis 

The first post assessments were administered in 2010. Prior to analysis, approximately 40 of the pre assess¬ 
ments were inadvertently discarded, leaving only 36 matched cases. As a result, we postponed the analysis 
until Spring 2011, giving a total of 82 matched pre and post assessments. We employed a graduate assistant 
to input the data into an EXCEL spreadsheet; another graduate assistant verified the accuracy of the entries 
and we monitored the process. Once completed, data were coded and then downloaded into SPSS Version 
19 for descriptive analysis. This report focuses on students’ pre and post program assessments relative to 
experience as measured across the six CPSELs. Data analyses included frequency distributions,matched case 
comparisons, T-tests, and a One-Way Analysis of Variance (ANOVA). 

4 Results 

Question 1: What degree of change occurred in students’ pre and post self assessment of their 
level of experience for each of the CPSELs, and how similar or different were the changes from 
pre to post self assessment among the six standards? 
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Table 2 summarizes the pre and post mean responses and computed difference (post test M - pre test 
M) illustrating the average change in candidates’ self reported level of experience for each CPSEL. As shown 
below, the mean differences between pre and post assessments ranged from .8 to 1.3 scale points. All 
differences were significant based on paired sample T-tests (p=.001). The largest gain (1.3) in self reported 
experience was for CPSEL 6 - influencing the political, social, economic, legal, and cultural context. The 
smallest gain (.8) was for Standard 2 - culture, instructional program. 

Table 2 

A Comparison of Pre and Post Self Assessments of ED AD Students over a Two-Year Period According 
to Difference in Mean Responses (Scale = 4 [high] to 1 [low] with N=82) 


California Professional S tandards for 

# Items 

Level of Experie 

nee 

Educational Leaders (CPSEL) 

Pre* M 

Post* M 

Difference** 

6 . Influencing political, social, economic, beat, and 
cultural conext 

9 

1.7 

3.0 

1.3 

5. Fairness, integrity... ethics 

10 

1.5 

2.7 

1.2 

3. Organizational management.. effective learning 
environment 

11 

1.8 

3.0 

1.2 

4. Collaboration., diverse community needs 

9 

2.0 

3.1 

1.1 

1. Vision of learning 

9 

2.2 

3 2 

1.0 

2. Culture, instructional program 

10 

2.2 

3.0 

.8 


Note: ‘Rounded to nearest tenth * *AB differences are significant (p=<001) 


Question 2: Did the degree of reported gain/loss in leadership experience vary among 
individual students? 

Mean differences are helpful in making group comparisons, but means can hide individual differences, 
and we were curious to know whether the reported gains/losses tended to be consistent among all students 
or varied. 

To answer this question, we conducted a matched score comparison and computed the frequency distri¬ 
butions of individual student’s mean response differences for each standard. We asked ourselves, what was 
the magnitude of this change and was growth evenly or disparately distributed? As shown in Table 2, the 
range of total group growth was from .8 to 1.3 full scale points. In order to discriminate more precisely in 
the matched case comparison, we calculated growth in .5 scale score point increments giving us a range of 
<.5 to >2.0 full scale points for analysis. Table 3 displays the findings from this analysis. The first statistical 
column (<.5) lists the percentage of students whose self reported level of experience grew less than one-half 
(.5) scale score points; the fifth statistical column (>2.0) lists the percentage of students whose self reported 
level of experience was equal to or greater than two scale score points. Because a full scale point gain on the 
4-point scale represents a25% gain, we appended an additional column. The last column is summative and 
displays the total percentage of candidates who reported gains of one full scale score point or more (>1.0). 

Table 3 

Matched Score Comparison of Pre and Post Self Assessment of Experience by Scale Score Points 
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Differences in Pre to Post Assessment 

California Professional S tandards for (Scale 4-1, High to Low) 


Educational Leaders 

N=82 

<S 

Sd 

.3-.99 
Sd 

1.0-13 

Sd 

1.51-1.99 

Sd 

=20 

Sd 

Total 

=10 

Sd 

6. Influencing political, social, economic, legal, and 
cultural context 

15 

15 

34 

19 

17 

70 

5. Fairness, integrity... ethics 

10 

27 

30 

11 

LL 

11 

63 

3. Organizational management , effective learning 
environment 

17 

20 

26 

CO 

9 

62 

1. Vision of learning 

17 

LL 

40 

12 

9 

61 

4. Collaboration., diverse community needs 

17 

24 

21 

23 

15 

59 

2. Culture, instructional pro sam 

26 

27 

32 

10 

6 

48 


Note: Percentages may exceed 100 due toroaiding 


Table 3 displays the six standards from highest to lowest according to the percentage of fieldwork partic¬ 
ipants indicating growth of at least one scale score point (1.0) between pre and post self-assessment. With 
this representation, three clusters of standards surfaced relative to growth of at least one scale score point 
(1.0). More students (70%) reported significant growth in their experiences understanding, responding to, 
and influencing the political, social, economic, legal, and cultural context of their environments (Standard 
6) than any of the other five standards. Approximately six of every 10 participants indicated at least a one 
scale score point improvement in Standards 5, 3, 1, and 4. 

The fewest number of participants (48%) self-assessed their growth as one scale point or more in pre post 
fieldwork experience in Standard 2 which states that an instructional leader promotes the success of every 
student by advocating, nurturing, and sustaining a school culture and instructional program conducive to 
student learning and staff professional growth. This finding is consistent with that shown in Table 2 which 
displayed total group mean growth. 

Question 3: How similar or different were the changes from pre to post assessment according 
to the total years of professional experience (TYE) of the candidates? 

The final question in this analysis focused on the relationship between reported gains/losses in perceived 
level of experience and TYE (total years experience, see Table 1). To answer this question, we compared 


average gains according to the following range of TYE, based on self reported data at the time of enrollment 
in the principal preparation program: 1-4 TYE; 5-9 TYE; or 10 or more TYE. Table 4 displays the average 
reported gains in experience for each standard. 

Table 4 

Pre and Post Self Assessment Mean Gains on Six CPSELS by Total Years of Experience 

California Professional S tandards for 
Educational Leaders (CPSELS) 

N=82 

1-4 TYE 
n = 33 

M 

5-9 TYE 
n=28 

M 

10+ TYE 
n = 21 

M 

1. Vision of learning 

1.06 

1.08 

1.07 

2. Culture, instructional program 

.77 

.86 

.97 

3. Organizational management... effective 

1.16 

1.07 

1.43 

learning environment 

4. Collaboration...diverse community needs 

1.11 

1.05 

1.31 

5. Fairness, integrity'... ethics 

1.12 

1.08 

1.47 

6. Influencing political, social, economic, legal, 
and cultural context 

1.22 

1.22 

1.36 


Note: Scale from High to Low, 4-1 


As shown in Table 4 and congruent with earlier comparisons, the least gains for all groups was relative 
to Standard 2 - culture, instructional program, although those with 10 or more years of experience reported 
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growth approaching one scale points (.97). With 18 points of comparison, three for each of the six standards, 
only three comparisons differed by more than two-tenths of a scale point. Each of these comparisons show 
candidates with the greatest number of years of experience (10+) reporting marginally greater growth than 
did those with fewer years of prior experience. The most experienced candidates reported greater growth 
than those in either of the two comparison groups, in Standard 3 - organizational management; Standard 4 
- collaboration.. .diverse community needs; and Standard 5 - fairness, integrity.. .ethics; - and Standard 6 - 
influencing political, social, economic, legal, and cultural context. A one-way ANOVA indicated no statistical 
significant differences among the three groups, based on an alpha of <.05. 

5 Conclusions and Discussion 

Comparisons of the pre and post self assessments of the 82 participants in this study support the conclusion 
that over the course of their administrative program participation, candidates perceived gains in their level of 
leadership experience. In that respect, our principal preparation program is fulfilling its purpose to provide 
these ‘administrators to be’ with experiences deemed appropriate for educational leaders in California. The 
greatest growth based on both mean growth differences and matched score comparisons, was for Standards 
6, 5, and 3. Standard 1 - vision of learning - was slightly ahead of Standard 4 - collaboration.. .diverse 
community needs - in the matched score comparison (Table 3) but somewhat behind relative to group mean 
growth as shown in Table 4. Our students appear to be saying, “I am growing in my ability to... 

.. .influence the political, social, economic, legal and cultural context (6); 

... act with fairness, integrity, and in an ethical manner (5); 

... ensure organizational management and an effective learning environment (3); 

... facilitate the development, articulation, implementation, and stewardship of a vision of learning that is 
shared and supported by all stakeholders (1); and 

.. .collaborate with faculty and community members, responding to diverse community interests and needs, 
and mobilizing community resources (4). 

In terms of meaningful differences, however, it was Standard 2 - culture, instructional program - that stood 
out. In every ranked comparison, Standard 2 was last, the only standard with a total mean difference of 
less than one scale point (.8) and the only standard with fewer than half (48%) of the candidates reporting 
growth of at least one scale score point . One explanation for the disparate results for Standard 2 may be 
found in the standard itself; it is the only standard to focus almost entirely on the instructional program: An 
instructional leader promotes the success of every student by advocating, nurturing, and sustaining a school 
culture and instructional program conducive to student learning and staff professional growth. Virtually all 
of the 82 participants were classroom teachers or had positions (e.g., teachers on special assignment) that 
focused on instruction. One can reasonably conclude that students enter the principal preparation program 
with more instructional experience than experience in any of the other five areas. Student learning is central 
to a teacher’s work and remains central to those who aspire to be instructional leaders in schools. 

Implications. Findings from this analysis support the value of a pre and post assessment instrument 
as a measure of candidate growth. Although dependent on the participants’ ability to self assess, these 
data provide useful information for program improvement purposes and compiled longitudinally will be 
increasingly valuable. Our second purpose in developing this tool was to individualize students’ fieldwork 
experiences based on their relative strengths and weaknesses. In order to come closer to that goal, future 
program enhancements might include the development of an interim assessment instrument to measure 
candidate progress through the program; analyzing candidates’ growth in experience somewhere in the 
middle of the program may provide us with more direction in counseling our students to become more 
experienced in other standards. We may also want to evaluate the experiences our candidates report when 
working with their mentors. We need to ensure that our candidates are provided opportunities based on 
need and not convenience. 

A next step is to test the validity of the Inventory by convening an expert group of practicing school 
leaders to review the items and make recommendations relative to the importance of their inclusion/exclusion 
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and possible addition of other activities. 

Limitations. We acknowledge three limitations to this study: a) Accuracy of the findings are dependent 
on the validity and reliability of the instrument, the Candidate Inventory; b) As a self assessment, the 
data reflect the subjective judgment of each candidate; and c) It is not possible to separate the actual 
impact of candidates’ fieldwork experience from the influence of their participation in other components 
of the administrative preparation program or from professional experiences unrelated to their credential 
preparation experiences. 
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